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Abstract. How can precise control be realised in intrinsically noisy systems? Here, we 

develop a general theoretical framework that provides a way to achieve precise control in 

^^ . signal-dependent noisy environments. When the control signal has Poisson or supra-Poisson 

noise, precise control is not possible. If, however, the control signal has sub-Poisson noise, 

then precise control is possible. For this case, the precise control solution is not a function, but 

\^ . a rapidly varying random process that must be averaged with respect to a governing probability 

^^ ' density functional. Our theoretical approach is applied to the control of straight-trajectory arm 

(-H ■ movement. Sub-Poisson noise in the control signal is shown to be capable of leading to precise 

control. Intriguingly, the control signal for this system has a natural counterpart, namely the 
bursting pulses of neurons -trains of Dirac-delta functions- in biological systems to achieve 
precise control performance. 
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1. Introduction 

Many mechanical and biological systems are controlled by signals which contain noise. This 
poses a problem. The noise apparently corrupts the control signal, thereby preventing precise 
control. However, precise control can be realised, despite the occurrence of noise, as has been 
demonstrated experimentally in biological systems. For example, in neural-motor control, 
as reported in [[1], the movement error is believed to be mainly due to inaccuracies of the 
neural-sensor system, and not associated with the neural-motor system. 

The minimum-variance principle proposed in [|2] |3l has greatly influenced the theoretical 
study of biological computation. Assuming the magnitude of the noise in a system depends 
strongly on the magnitude of the signal, the conclusion of [l2l [3]| is that a biological system is 
controlled by minimising the execution error. 

A key feature of the control signal in a biological system is that biological computation 
often only takes on a finite number of values. For example, 'bursting' neuronal pulses in 
the neural-motor system control seem very likely to have only three states, namely inactive, 
excited, and inhibited. This kind of signal (neuronal pulses) can be abstracted as a dynamic 
trajectory which is zero for most of the time, but intermittently takes a very large value. 
Generally, this kind of signal looks like a train of irregularly spaced Dirac-delta functions. In 
this work we shall theoretically investigate the way signals in realistic biological systems are 
associated with precise control performance. We shall use bursting neuronal pulse trains as a 
prototypical example of this phenomenon. 

In a biological system, noise is believed to be inevitable and essential; it is a part of a 
biological signal and, for example, the magnitude of the noise typically depends strongly on 
the magnitude of the signal |l2l[3]|. One characteristic of the noise in a system is the dispersion 
index, a, which describes the statistical regularity of the control signal. When the variance in 
the control signal is proportional to the 2a-th power of the mean control signal, the dispersion 
index of the control noise is said to be a. It was shown in |l2l |3l and elsewhere (e.g., flU |5l) 
that an optimal solution of analytic form can be found when the stochastic control signal 
is supra- Poisson, i.e., when a > 0.5. However, the resulting control is not precise and a 
non-zero execution error arises. In recent papers, a novel approach was proposed to find the 
optimal solution for control of a neural membrane []6], and a model of saccadic eye movement 
|I3. It was shown that if the noise of the control signal is more regular than Poisson process 
(i.e., if it is sub-Poisson, with a < 0.5), then the execution error can be shown to reduce 
towards zero [|6l|71. This work employed the theory of Young measures [[T3l[T4| . and involved 
a very specific sort of solution (a 'relaxed optimal parameterized measure solution'). We 
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note that many biological signals are more regular than a Poisson process: e.g., within in- 
vivo experiments, it has often been observed that neuronal pulse signals are sub-Poisson in 
character (a < 0.5) lfT5l[T6l . However, in [[6113, only a one-dimensional linear model was 
studied in detail. Thus the results and methods cannot be applied to the control of general 
dynamical systems. The work of flUITl however, leads to a much harder problem: the general 
mathematical link between the regularity of the signal's noise and the control performance 
that can be achieved. 

In the present work we establish some general mathematical principles linking the 
regularity of the noise in a control signal with the precision of the resulting control 
performance, for general nonlinear dynamical systems of high dimension. We establish a 
general theoretical framework that yields precise control from a noisy controller using modem 
mathematical tools. The control signal is formulated as a Gaussian (random) process with a 
signal-dependent variance. Our results show that if the control signal is more regular than a 
Poisson process (i.e., if a < 0.5), then the control optimisation problem naturally involves 
solutions with a specific singular character (parameterized measure optimal solutions), which 
can achieve precise control performance. In other words, we show how to achieve results 
where the variance in control performance can be made arbitrarily small. This is in clear 
contrast to the situation where the control signals are Poisson or more random than Poisson 
(a > 0.5), where the optimal control signal is an ordinary function, not a parameterized 
measure, and the variance in control performance does not approach zero. The new results 
can be applied to a large class of control problems in nonlinear dynamical systems of high 
dimension. We shall illustrate the new sort of solutions with an example of neural-motor 
control, given by the control of straight-trajectory arm movements, where neural pulses act as 
the control signals. We show how pulse trains may be realised in nature which lead towards 
the optimisation of control performance. 

2. Model and Mathematical Formulation 

To establish a theoretical approach to the problem of noisy control, we shall consider the 

following general system 

dx 

— = a{x{t),t)+b{x{t),t)u{t) (1) 

where: t is time (t > 0), x{t) = [xi{t), . . . ,Xn{t)]'^ is a column vector of 'coordinates' 
describing the state of the system to be controlled (a T-superscript denotes transpose) and 
u{t) = [ui{t), . . . , Um]'^ , is a column vector of the signals used to control the x system. The 
dynamical behaviour of the x system, in the absence of a control signal, is determined by 
a(x,t) and 6(x,t), where a(x,t) consists of n functions: a{x,t) = [ai{x,t), . . . ,an{x,t)]^ 
and b{x, t) isannxm 'gain matrix' with elements bij{x, t). The system ([U) is a generalisation 
of the dynamical systems studied in the literature [|2l |3l |6l 13. 

As stated above, the control signal, u{t), contains noise. We follow Harris's work [|2l|3l 
on signal-dependent noise theory by modelling the components of the control signal as 

Uiit) = Ai(t) + Ut) (2) 
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where Aj(t) is the mean control signal at time t of the i'th component of u(t) and all noise 
(randomness) is contained in C,i{t). In particular, we take the C,i{t) to be an independent 
Gaussian white noises obeying E [C,i{t)] = 0, E [Cj(^)Ci(^')] = (yi{t)o'j{t')5{t — t')Sij where 
E [■] denotes expectation, S{-) is Dirac-delta function, and 5ij is Kronecker delta. The 
quantities crj(t), which play the role of standard deviations of the 0(^)5 arc taken to explicitly 
depend on the mean magnitudes of the control signals: 

a,it) = ^\{t)r (3) 

where Ki is a positive constant and a is the dispersion index of the control process (described 
above). 

Thus, we can formulate the dynamical system, Eq. ([D, as a system of Ito diffusion 
equations: 

dx = A{x,t,X)dt + B{x,t,X)dWt (4) 

where: (i) Wt = [Wi^t, ■ ■ ■ ,Wm,t]~^ contains m independent standard Wiener processes; 
(ii) the quantity A{x, t, A) denotes the column vector [Ai(x, t, A), . . . , A„(x, t, A)]^, the i'th 
component of which has the form Ai(x, t, A) = aj(x, t) + ^^^^ hji^, t)\j\ (iii) the quantity 
B(x,t,X) is the matrix, the z,j'th element of which is given by _Bjj(x,t, A) = bij{x,t)Kj\\j\°' 
where i = 1,- ■ ■ ,n and j = 1, . . . , m. We make the assumption that the range of each ^j is 
bounded: -My < \i < My with My a positive constant. Let Vl = [-My, My]"^ be the 
region where the control signal takes values, with "^ denoting the m-order Cartesian product. 
Let H be state space of x. In this paper we assume it be bounded. 

Let us now introduce the function </)(x, t) = [01 (x, t), ■ ■ ■ , (pki^, t)]^, which represents 
the objective that is to be controlled and optimised. For example, for a linear output we can 
take 0(x, t) = Cx for some kyin matrix C; in the case that we control the magnitude of x, we 
can take 0(x, t) = \\x\\2', we may even allow dependence on time, for example if the output 
decays exponentially with time, we can take exp(— 7t)a;(t) for some constant 7 > 0. 

The aim of the control problem we consider here is: (i) to ensure the expected trajectory 
of the objective (j){x{t),t) reaches a specified target at a given time, T, and (ii) to minimise the 
execution error accumulated by the system, during the time, R, that the system is required to 
spend at the 'target' [l2l|3l|6l|7l[8l|9l[l0l[II]. In the present context, we take the motion to start 
at time t = and subject to the initial condition a;(0). The target has coordinates z{t) and we 
need to choose the controller, u{t), so that for the time interval T < t < T + R the expected 
state of the objective (f){x{t),t) of the system satisfies E[0(x(t), t)] = z(t). The accumulated 
execution error is J^ ^^ var {(j)i{x{t)^t)) dt and we require this to be minimised. 

Statistical properties of x{t) can be written in terms of p(x, t), the probability density of 
the state of the system ([4]) at time t, which satisfies the Fokker-Plank equation 

^^^ = £op;p(x,0) = 5(x-x(0)), te[0,T + i?] (5) 

with 

d[ai{x, t) + X;^i hj{,x, t)Xj{t))p{x, t)] 



Cop=-Y^ 



dxi 
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1 ^ s'[Y:Zi4>>dx,t)bit(x,t)\\(t)?'p(^,t)} 



+ iE 



«j=i 



dxidxj 



Three important quantities are the following: 

(A) The accumulated execution error: Jj, J~\\(j){x,t) — z{t)\\ p{x,t)dxdt; 

(B) The expectation condition on x{t): J„(l){x,t)p{x,t)dx = z{t), for all t in the interval 
T <t<R + T; 

(C) The dynamical equation of p(x, t) described as (|5]). 



3. The Young Measure Optimal Solution 

To illustrate the idea of the solutions we introduce here, namely Young measure optimal 
solutions, we provide a simple example. Consider the situation where x and u are one- 
dimensional functions, while a(x, t) = px, b(x, t) = q, k = 1, z(t) = zq and 0(x, t) = x. 
Thus (dl) becomes 

dx 

-— = px + qu. (6) 

dt 

This has the solution x{t) = xoexp{pt) + J^ exp(p(t — s))qX{s)ds + J^ exp(p(t — 
s))q\X{s)\'^dWs. Thus, its expectation is E(x(t)) = Xoexp(j9t) + J^ exp{p{t — s))qX{s)ds and 
its variance is var{x{t)) = J^ exp(2p(t — s))q'^\X{s)\'^°'ds. The solution of the optimisation 
problem is the minimum of the following functional: 

H[X]= / / exp{2p{t - s))q^\X{s)\^''dsdt 



+ 



T+R 



T 
T+R 



7(t)[xoexp(pt) + / exp(p(t — s))qX{s)ds — zq] > dt 



{[g{t)\X{t)r-mX{t)]+f,{t)}dt 



(V) 



with 



9{t) 



fit) 



31 

2p 



t <T 
T+R> t>T 



exp(2p(T + R-t))- exp(2p(T - t)) 
exp{2p{T + R - t)) - I 

~ It q'j{s) exp{p{s — t))ds t <T 
- !™ ?7(^) exp(p(s - t))ds T + R>t>T 
jj,(t) = 7(t)[xoexp(pt) - zo], 

for some integrable function 7(t), which serves as a Lagrange auxiliary multiplier. 

In the general case, we minimise (A) using (B) and (C) as constraints via the introduction 
of appropriate x and t dependent Lagrange multipliers. This leads to a functional of the mean 
control signal, H [A], with the form H[X] = jj, h{t, X{t))dt (see below and Appendix A). 
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Let us use ^ = [^i, ■ ■ ■ , im\^ to denote the value of A(t) at a given time of t, i.e., ^ = A(t); ^ 
will serve as a variable of the Young measure (see below). We find 

m 

h{t,o = Y.^giitmi'"' - Mm + z{t) (8) 

where gi{t), fi{t) and z{t) are functions with respect to t but are independent of the variable 

i- 

The abstract Hamiltonian minimum (maximum) principle (AHMP) [[T2l| provides a 
necessary condition for the optimal solution of minimising (A) with (B) and (C), which is 
composed of the points in the domain of definition of A, namely, Vt, that minimize the function 
h{t, ^) in dH]), at each time, t, which is named Hamiltonian integrand. This principle tells us 
that the optimal solution should pick values of the minimum of h(t, ^) with respect to ^, for 
each t. 

If the control signal is supra-Poisson or Poisson, namely the dispersion index a > 0.5, 
for each t E [0,T + R], the Hamiltonian integrand h{t, ^) is convex (or semi-convex) with 
respect to ^ and so has a unique minimum point with respect to each ^j. So, the optimal 
solution is a deterministic /wncrion of time: for each to, Ai(to) can be regarded as picking 
value at the minimum point of h{t, ^) for t = to- 

When a < 1/2, namely when the control signal is sub-Poisson, it follows that h{t,^) 
is no longer a convex function. Figs. \T\ show the possible minimum points of the term 
5'i(^)l6P° ~ fiit)(i with (yfj(t) > and /j(t) > 0. From the assumption that the range of 
each ^i is bounded, namely —My < ^i < My, it then directly follows, from the form of 
h{t,^), that the value of ^i which optimises h{t,^) is not unique; there are three possible 
minimum values: —My, 0, and My, as shown in TablefT] So, no explicit function A(t) exists 
which is the optimal solution of the optimisation problem (A)-(C). However, an infinimum of 
([8]) does exist. 

Proceeding intuitively, we first make an arbitrary choice of one of the three optimal 
values for ^i (namely one of —My, 0, and My) and then average over all possible choices 
at each time. With 'r]t,iiO the probability density of ^i at time t, the average is carried out 
using the distribution (probability density functional) ?7[A] oc Ylt iVtAO which represents 
independent choices of the control signal at each time. Thus, for example, the functional 
H[X\ becomes functionally averaged over A(-) according to J^ (^J h{t, X)r][X]d[X\) dt. 
The optimisation problem has thus shifted from determining a function (as required when 
« > 1/2) to determming a probability density functional, 'qlX]. This intuitively motivated 
procedure is confirmed by optimisation theory- and this leads us to Young measure theory. 

Let us spell it out in a mathematical way. Young measure theory lfT3l [T4| provides 
a solution to an optimization problem where a solution, which was a function, becomes 
a linear functional of a parameterized measure. By way of explanation, a function, A(t), 
yields a single value for each t, but a parameterized measure {rit{-)} yields a set of values 
on which a measure (i.e., a weighting) rit{-) is defined for each t. Afunctional with respect 
to a parameterized measure can be treated in a similar way to a solution that is an explicit 
function, by averaging over the set of values of the parameterized measure at each t. In detail. 
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a functional of the form H[\] = J^ h{t, X{t))dt, of an explicit function, X{t), can have its 
definition extended to a parameterized measure 'qt^-), namely H[7]] = J^ J^ h{t, ^)r]t{d^)dt. 
In this sense, an explicit function can be regarded as a special solution that is a 'parameterized 
concentrated measure' (i.e., involving a Dirac-delta function) in that we can write H[X] = 
/o /n ^(^''^)'^('^ ~ \{t))d$,dt. Thus, we can make the equivalence between the explicit 
function A(t) and a parameterized concentrated measure {6{^ — \{t))}t and then replace this 
concentrated measure, when appropriate, by a Young measure. 

Technically, a Young measure is a class of parameterized measures that are relatively 
weak*-compact such that the Lebesgue function space can be regarded as its dense subset in 
the way mentioned above. Thus, by enlarging the solution space from the function space to 
the (larger) Young measure space, we can find a solution in the larger space and the minimum 
value of the optimisation problem, in the Young measure space, coincides with the infinimum 
in the Lebesgue function space. 

Forany function r(a;,t,.^), we denote a symbol ■ as theinnerproductof r(x, t, ,^) over the 
parameterized measure T]t{d^), by averaging r(x, t, ^) with respect to ^ via rit{-). That is we 
define r(x, t, "'?< to represent J^ r(x, t, ^)r]t{d^). In this way we can rewrite the optimisation 
problem (A)-(C) as: 

min^ fj. f^\\(f){x,t) — z(t)\\'^p{x,t)dxdt 

subject to ^^^ = [{C-ri)op]{x,t), on[0,T]xE, p{x,0) =po{x), 

xeE,te[0,T + R] 

/„ (f){x,t)p{x,t)dx = z{t), on[T,T + R], 77 G y. 
Here, y denotes the Young measure space, which is defined on the state space S with 
t E [0,T + R], while r] = {r]t{-)} denotes a shorthand for the Young measure associated 
with control; {C ■ 77) o p is defined as 

[C.r]]op= / C{t,x,^)op{x,t)r]t{d^) 
J n 

d[A{x,t,Op{x,t)] 



l^ 



^ dxi 

d''{[B{x,t,i)B'^{x,t,i)]i,p{x,t)} 



1 

iiiii I /— 1/1 II I /— ii-'»iii ii>- I 

mm 



2 f-^^ dxidxj 

d[ai{x, t) + X;"Li hj{x, t)Q)p{x, t)] 



E 



dxi 



So, we can study the relaxation problem ^ instead of the original one, (A)-(C). We assume 
that the constraints in ^ admit a nonempty set of A(t), which guarantees that the problem (|9]) 
has a solution. We also assume the existence and uniqueness of the Cauchy problem of the 
Fokker-Plank equation (|5]). 

The abstract Hamiltonian minimum (maximum) principle (Theorem 4.1.17 [[T2l ) also 
provides a similar necessary condition for the Young measure solution of Q, if it admits a 
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solution, that is composed of the points in Vt which minimise the integrand of the underlying 
'abstract Hamiltonian' . By employing variational calculus with respect to the Young measure, 
we can derive the form (|8]), for the Hamiltonian integrand. See Appendix A for details. 

Via this principle, the problem conceptively reduces to finding the minimum points of 
h{t,^). From Table [H for a sufficiently large My, it can be seen that, if a < 0.5, then 
the minimum points for each t with gi{t) > may be TWO points {0, My} or {—My, 0}. 
Hence, in the case of a < 0.5, the optimal solution of ([91) is a measure on {My, 0} or 
{—My, 0}. This implies that the optimal solution of ^ should have the following form 
Vti') = Vi,ti') ^ 5 ■ ■ ■ 5 1m.,t{), where x stands for the Cartesian product, and each ?7j ^ we adopt 
is a measure on {—My, 0, My}: 

7]i^t{-) = l^i{t)6My{-) + jyi{t)6^My{-) + [1 - |^^{t) - iyi{t)]6o{-) (10) 

where /ii(t) and z/j(t) are non-negative weight functions. The optimisation problem 

corresponds to the determination of the /ii(t) and ^'j(t). Averaging with respect to 7] 

corresponds to the optimal control signal when the noise is sub-Poisson (a < 0.5). This 

assignment of a probability density for the solution at each time is known in the mathematical 

literature as a Young Measure [[T2l [T3l [T4l . For all i and t, the weight functions satisfy: (i) 

/ij(t) + z/j(t) < 1 and (ii) iii{t)Vi{t) = (owing to the properties mentioned above that h{t, ^i) 

cannot simultaneously have both My and —My as optimal). 

Consider the simple one-dimensional system ^. We shall provide the explicit form of 

the optimal control signal u{t) as a Young measure. Taking expectation for both sides in (|6]), 

we have 

(iE (x) -^ / X w N 
-^=pE{x) + q\{t). 

Since we only minimise the variance in [T, T + R] for some T > and R > 0, the control 
signal u{t) for t e [0, T) is picked so that the expectation of x{t) can reach zq at the time 
t = T. After some simple calculations, we find a deterministic A(t) as follows: 

X{t) = exp(p(-T + t , te [0, T] 

Tq 

such that E(x(T)) = zq. Then we pick A(t) = -pzo/q for t e [T,T + R] such that 
dE{x{t))/dt = for all t e [T,T + R]. Hence, E(x(t)) = zo for all t e [T,T + R]. In 
the interval [T, T + R], as discussed above, for a sufficiently large My, the optimal solution 
of A(t) should be a Young measure that picks values in {0, My, —My}. To sum up, we can 
construct the optimal A(t) as follows: 

f 5A(t)(-) te[0,T) 

Vt{-) = { ^M.(-)^ + ^o(-)[l + ^] te[T,T + R]ii -pzo/q>Oor 
[ ^-A/.-(-)^ + ^o(-)[l-^] te[T,T + R]iipzo/q>0. 
It can be seen that in [0, T), r]t{-) is in fact a deterministic function as the same as A(t). 
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4. Precise Control Performance 

We now illustrate the control performance when the noise is sub-Poisson. For the general 
nonlinear system ([T), we cannot obtain an explicit expression for the probability density 
functional 77 [A], Eq. (flOl) . or the value of the variance (execution error). However, we can 
adopt a non optimal probability density functional which illustrates the property of the exact 
system, that the execution error becomes arbitrarily small when the bound of the control 
signal. My, becomes arbitrarily large. In the simple case ©, we note that if there is a u{t), 
such that Fi(x{t)) = z(t), then the variance becomes, expressed by Young measure fi{-), 

var{x{t))= I I exp{2p{t - s))q^\^\^'^7i{dOds 

r2aHs)\ 



J -My 
t 



exp{2p{t~s))q'Mf^^ds, 
Iq My 

which converges to zero as My — )■ 00, due to a < 0.5. That is, the minimised execution error 

can be arbitrarily small if the bound of the control signal. My, goes sufficiently large. 

In fact, this phenomenon holds for general cases. The non optimal probability density 

functional is motivated by assuming that there is a deterministic control signal u(t) which 

controls the dynamical system 

U/X 

— = A{x,t,u{t)) (11) 

which is the original system ([I]), with the noise removed. The deterministic control signal u{t) 
causes x{t) to precisely achieve the target trajectory x{t) = z(t) for T < t < T + R. 

Then, we add the noise with the signal-dependent variance: ai = K,i\Xi\°' with some a < 
0.5, which leads a stochastic differential equation, dx = A{x, t, X{t))dt + B{x, t, X{t))dWt. 
The non optimal probability density that is appropriate for time t, namely 57(j(^), is 
constructed to have a mean over the control values {—My, 0, My}, which equals u{t). This 
probability density is 

ViA^i) = -j^^miYiX^) + (1 - -J^miXi) (12) 

where a{t) = sign{ui{t)) and, by definition, Ui{t) = j_^^ XifjiAXi)dXi. We establish in 
Appendix B that the expectation condition ((B) above) holds asymptotically when My — )■ 00, 
which shows that the non optimal probability density functional is appropriately 'close' to 
the optimal functional. The accumulated execution error associated with the non optimal 
functional is estimated as 



mm y y™var[x(t))]cit = o[-^^) (13) 

and, in this way, optimal performance of control, with sub-Poisson noise, can be seen to 
become precise as My is made large. By contrast, if « > 0.5, the accumulated execution 
error is always greater than some positive constant. 

To gain an intuitive understanding of why the effects of noise are eliminated for a < 0.5 
we discretise the time t into small bins of identical size At. Using the 'noiseless control' Ui{t), 
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we divide the time bin [t, t + At] into two complementary intervals: [t, t + \u{t)\^t / My] and 
[t + |n(t)|At/My, t + At], and assign Aj = cr(t)My for the first interval and A^ = for 
the second. When At — ;• the effect of the control signal Aj(t) on the system approaches 
that of Mj(t), although Aj(t) and Ui{t) are quite different. The variance of the noise in the 
first interval is KiMy" and is in the second. Hence, the overall noise effect of the bin is 



(77 



Ki\Ui{t)\ 

My 



■ M|-° 



K^\Ui{t)\M. 



2a-l 
Y 



Remarkably, this tends to zero as My -^ oo if 



a < 1/2 (i.e., for sub-Poisson noise). The discretisation presented may be regarded as a 
formal stochastic realisation of the probability density functional (Young measure) adopted. 
The interpretation above can be verified in a rigorous mathematical way. See Appendix B for 
details. 



5. Application and Example 

Let us now consider an application of this work: the control of straight-trajectory arm 
movement, which has been widely studied |[8l |9l [TOl |TT]| and applied to robotic control. 
The dynamics of such structures are often formalised in terms of coordinate transformations. 
Nonlinearity arises from the geometry of the joints. The change in spatial location of the 
hand that results from bending the elbow depends not only on the amplitude of the elbow 
movement, but also on the state of the shoulder joint. 

For simplicity, we ignore gravity and viscous forces, and only consider the movement of 
a hand on a horizontal plane in the absence of friction. Let 9i denote the angle between the 
upper arm and horizontal direction, and 02 be the angle between the forearm and upper arm 
(Fig. |2]). The relation between the position of hand [xi, X2] and the angles [9i, 62] is 

9i = arctan(x2/xi) — arctan(/2 sin6'2/(/i + I2 00862)) 

62 = arccos[(xi + xj - ij - ll)/{2lil2)], 
where /i 2 are moments of inertia with respect to the center of mass, for the upper arm and 
forearm. When moving a hand between two points, a human maneuvers their arm so as 
to make the hand move in roughly a straight line between the end points. We use this to 
motivate the model by applying geostatics theory [8J. This implies that the arm satisfies an 
Euler-Lagrange equation, which can be described as the following nonlinear two-dimensional 
system of differential equations: 



dm 

In these equations 

N = 



IT 

2' 



f C{9i, 62, 61,62 

62(0) = ^, ^l(0)=^2(0) 



02 



7o 



Qi 

Q2 



(14) 



Ji + TJiirj + 7722/1 
+I2 + m2rj + 2k cos 62 
I2 + m2r| + k cos 62 



C = k sin 6*2 



62 61 
01 



0,. 



I2 + ?722r2 + k cos 02 

1 2 + m2rl 



Q, = A,(t) + Ko|A,(t)r- 



cit 
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where rrii, li, and Jj are, respectively the mass, length, and moment of inertia with respect to 
the center of mass for the i'th part of the system and i = 1 (i = 2) denotes the upper arm 
(forearm), ri 2 are the lengths of the upper- and fore-arms, and 70 is the scale parameter of the 
force. Additionally, k = m2/ir2, while Ai2(t) are the means of two torques (51,2(^)5 which 
are motor commands to the joints. The torques are accompanied by signal-dependent noises. 
All other quantities are fixed parameters. See [Bj for the full details of the model. The values 
of the parameters we pick here are listed in Table 

For this example, we shall aim to control the hand such that it starts at t = 0, 
with the initial condition of (fT4l) . reaches the target at coordinates H = [Hi,H2] at time 
t = T, and then stays at this target for a time interval of R. We use the minimum 
variance principle to determine the optimal task, which is more advantageous than other 
optimisation criteria to control a robot arm [|8] HI]. Let [xi{t) , X2{t)] be the Cartesian 
coordinates of the hand that follow from the angles [9i{t),92{t)]. The minimum variance 
principle determines minAi,A2 Jt [var(xi(t)) + var{x2{t))]dt, subject to the constraint that 
E[xi(t), X2(t)] = [Hi, H2]{orT <t<T + R, with -My < Xi < My- Despite not being in 
possession of an explicit analytic solution, we can conclude that if a > 0.5, the optimisation 
problem results from the unique minimum to the Hamiltonian integrand and hence yields 
Ai(t) and A2(t) which are ordinary functions. However, if a < 0.5, the optimal solution 
of the optimisation problem follows from a probability density functional analogous to Eq. 
(flOl) (i.e., a Young measure over Aj G {—My, 0, My}). Thus, we can relax the optimisation 
problem via Young measure as follows: 

Q^= j ^ U^ + K^\UtTdW,/dt) ■ r]i,t{dO^ ^ = 1.2, 

J -My ^ ' 

and 

min^^2(.) J^ [var(xi(t))^ + var[a;(2(t))^](it 

Subject to E[xi(T), X2(T)] = [i^i, H^, t ^ [T, T + i?] (15) 

i,^{-My,My\. 

We used Euler's method to conduct numerical computations, with a time step of 0.01 
msec in (fT4l) . This yields a dynamic programming problem (see Methods). Fig. [3] shows the 
means of the optimal control signals Ai 2(t) with a = 0.25 and My = 20000: 

pMy 

Ut)= / ^V^AdO, ^ = 1,2. 

J -My 

According to the form of the optimal Young measure, the optimal solution should be 

ViAds) = 



^5m,(s) + (1 - ~^)6ois) ds Ut) > 



My "My\^I ' V-" My 

^^5.My{s) + {l-^^)5o{s)\ds Ut)<0 

6o{s)ds otherwise. 

It can be shown (derivation not given in this work) that in the absence of the noise term, 
the arm can be accurately controlled to reach a given target for any T > 0. In this case. Fig. |4] 
shows the dynamics of the angles, their velocities, and accelerators, in the controlled system. 
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removed noise. See, in comparison, the dynamical system with noise, whose dynamics of the 
angles, velocities, and accelerations are illustrated in Fig. |5l and its dynamics are exactly the 
same as those in the case with noise removed. However, the acceleration dynamics of a noisy 
dynamic system appear discontinuous since the control signals, that have noises and are added 
to the right-hand sides of the mechanical equations, are discontinuous (noisy) in a numerical 
realisation. However, according to the theory of stochastic differential equations [fTTl . (fT4] | 
has continuous solution. Hence, these discontinuous acceleration dynamics lead very smooth 
dynamics of velocities and angles, as shown in Fig. [51 

Figs. [6] (a) and (b) illustrate that the probability density functional, for this problem, 
contains optimal control signals that are similar to neural pulses. Despite the optimal solution 
not being an ordinary function when a < 0.5, the trajectories of the angles 6i and 62 of the 
arm appear quite smooth, as shown in Fig. |5](a), and the target is reached very precisely if 
the value of My is large. By comparison, when a > 0.5 the outcome has a standard deviation 
between 4 to 6 cm, which may lead to a failure to reach the target. A direct comparison 
between the execution error of the cases a = 0.8(> 0.5) and a = 0.25(< 0.5) is shown in the 
supplementary movies (supplementary videos 'Video SI' and 'Video S2') of arm movements 
of both cases. Our conclusion is that a Young measure optimal solution, in the case of sub- 
Poisson control signals, can realize a precise control performance even in the presence of 
noise. However, Poisson or Supra-Poisson control signals cannot realise a precise control 
performance, despite the existence of an explicit optimal solution in this case. Thus a < 0.5 
significantly reduces execution error compared with a > 0.5. 

With different T (the starting time of reaching the target) and R (the duration of reaching 
the target), under sub-Poisson noise, i.e., a < 0.5, the system can be precisely controlled by 
optimal Young measure signals with a sufficiently large My- Since the target in the reachable 
region of the arm, it implies that the original differential system of (fT4l) with the noise removed 
can be controlled for any T > and i? > (HHH. According to the discussion in Appendix 
B (Theorem O, the execution error can be arbitrarily small when My is sufficiently large. 
However, for a smaller T, i.e., the more rapid the control is, the larger means of the control 
signals will be. As for the duration R, by picking the control signals as fixed values (zeros in 
this example) such that the velocities keep zeros, the arm will stay at the target for arbitrarily 
long or short. Similarly, with a large My, the error (variance) of staying at the target can be 
very small. To illustrate these arguments, we take T = 100 (msec) and R = 100 (msec) for 
example (all other parameters are the same as above). Fig. |7] shows that the means of the 
optimal Young measure control signals before reaching the target have larger amplitudes than 
those when T = 650 (msec) and Fig. [8] shows that the arm can be precisely controlled to 
reach and stay at the target. 

The movement error depends strongly on the value of the dispersion index, a, and the 
bound of the control signal. My. Fig. [9] indicates a quantitative difference in the execution 
error between the two cases a < 0.5 and a > 0.5, if a is close to (but less than) 0.5. The 
execution error can be appreciable unless a large My is used. For example if a = 0.45, as 
in Fig. [9l the square root of the execution error is approximately 0.6 cm when My = 20000. 
From (fT3l) . the error decreases as My increases, behaving approximately as a power-law, as 
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illustrated in the inner plot of Fig. |9l The logarithm of the square root of the execution error is 
found to depend approximately linearly on the logarithm of My when a = 0.25, with a slope 
close to —0.25, in good agreement with the theoretical estimate (fT3T ). 

We note that in a biological context, a set of neuronal pulse trains can achieve precise 
control in the presence of noise. This could be a natural way to approximately implement 
the probability density functional when a < 0.5. All other parameters are the same as 
above {a = 0.25). The firing rates are illustrated in Fig. |6](a) and (b) and broadly coincide 
with the probability density functional we have discussed. In particular, at each time t, the 
probability ?7j ^ can be approximated by the fraction of the neurons that are firing, with the 
mean firing rates equal the means of the control signals (see Methods). The approximations 
of the components of the noisy control signals are shown in Figs. [IO](a) and (b) respectively. 
Fig{TO](c) and (d) illustrate such an implementation of the optimal solution by neuronal pulse 
trains. Using the pulse trains as control signals, we can realise precise movement control. 
We enclose two videos 'movieUP.avi' and 'movieDOWN.avi' to demonstrate the efficiency 
of the control by pulse trains with two different targets. As they show, the targets are precisely 
accessed by the arm. We point out that the larger the ensemble is, the more precise the 
control performance will be, because a large number of the neurons in an ensemble can 
theoretically lead to a large My as we mentioned above, which results in an improvement 
of the approximation of a Young measure and decreases the execution error as stated in (IT3T ). 

We note that these kinds of pattems of pulse trains have been widely reported in 
experiments, for example, the synchronous neural bursting reported in [il8i . This may provide 
a mathematical rationale for the nervous system to adopt pulse-like signals to realise motor 
control. 

6. Conclusions 

In this paper, we have provided a general mathematical framework for controlling a class 
of stochastic dynamical systems with random control signals whose noisy variance can be 
regarded as a function of the signal magnitude. If the dispersion index, a, is < 0.5, which is 
the case when the control signal is sub-Poisson, an optimal solution of explicit function does 
not exist but has to be replaced by a Young measure solution. This parameterized measure can 
lead a precise control performance, where the controlling error can become arbitrarily small. 
We have illustrated this theoretical result via a widely-studied problem of arm movement 
control. 

In the control problem of biological and robotic systems, large control signals are needed 
for rapid movement control [[2T| . When noise occurs, this will cause imprecision in the control 
performance. As pointed out in [|9l |22l |23l |24l |221, a trade-off should be considered when 
conducting rapid control with noises. In this paper, we still use a "large" control signal but 
with different contexts. With sub-Poisson noises, we proved that a sufficiently large My, i.e., 
a sufficiently large region of the control signal values, can lead precise control performance. 
Hence, a large region of control signal values plays a crucial role in realising precise control in 
noisy environments, for both "slow" and "rapid" movement control. In numerical examples. 
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the larger My we pick, the smaller control error will be, as shown in the inset plot of Fig. |9] 
as well as (fT3l) (Theorem |2] in Appendix B). 

Implementation of the Young measure approach in biological control appears to be a 
natural way to achieve precise execution error in the presence of sub-Poisson noise. In 
particular, in the neural-motor control example illustrated above, the optimal solution in the 
case of a < 0.5 is quite interesting. Assume we have an ensemble of neurons which fire pulses 
synchronously within a sequence of non overlapping time windows, as depicted in Figs. [6]C 
and D. We see that the firing neurons yield control signals which are very close, in form, to 
the type of Young measure solution. This conclusion may provide a mathematical rationale 
for the nervous system why to adopt pulse-like trains to realise motor control. Additionally, 
we point out that, our approach may have significant ramifications in other fields, including 
robot motor control and sparse functional estimation, which are issues of our future research. 

Methods 

Numerical methods for the optimisation solution. We used Euler's method to conduct 
numerical computations, with a time step of At = 0.01 msec in (fT4)) with a < 0.5. This yields 
a dynamic programming problem. First, we divide the time domain [0, T] into small time bins 
with a small size At. Then, we regard the process rji^t in each time bin [nAt, [n + l)At] 
as a static measure variable. Thus, the solution reduces to finding two series of nonnegative 
parameters yUj „ and Ui^n with yUj „ + i^^ „ < 1 and fii^n^i^n = such that 

{My nAt <t < {n + /ii,„) At 
-My (n + /ii,„)At < t < (n + ^i,„ + i^i,„)At . 
(n + /ii,„ + i/i,„)At <t < (n + l)At. 

The approximate solution of the optimisation problem requires nonnegative /ij „ and Ui^n that 
minimise the final movement errors. We thus have a dynamic programming problem. We 
should point out that in the literature, a similar method was proposed to solve the optimisation 
problem in a discrete system with control signals taking only two values [[TTl [T9ll20l . Thus, 
the dynamical system ^ becomes the following difference equations via the Euler method: 

m 

Xi{k + 1) = Xi{k) + Atlai{x{k),tk) + '^bij{x{k),tk)[iJ,j,k - i^j,k]My 

m 

+ ^ hij{x{k),tk)Kj\fAt{iij^k + yj,k)MyVj, /c = 0, 1, 2, ■ ■ ■ , 

where Uj, j = 1, . . . , m, are independent standard Gaussian random variables. We can derive 
difference equations for the expectations and variances of x{k), by ignoring the higher order 
terms with respect to At: 



E(xi(A; + 1)) = E(x,(A;)) + At|E[a(x(A;), t^)] 

m 

+ 2]]E[%(x(A:),tfc)][/ij-fe - Vj^k]My 
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cov(xj(A; + 1), Xj/(/c + 1)) = cov(xj(A;), Xi'{k)) + Atcov( Xj/(/c), {ai{x{k),tk) 

m 

+ ^bij{x{k),tk)[iJ.j,k - iyj,k]MY} 

m 

+ Atcov(^Xi{k), {ai/(x(/c),4) + ^ 6i'j(a;(/c),4)[/ij,fc - Uj^kWr} 

m 

+ At ^ cov {bi>j{x{k),tk), bij{x{k),tk)){iJ,j,k + Uj,k)MY". 

Thus, Eq. (|9l) becomes the following discrete optimization problem: 

min;,,,fe,i.,,fc EfcVar(0(x(A;),tfe)) 

subject to E(0(x(/c),tfc)) = z{tk), fii,k + i^i,fc < 1, (16) 

/ii,fc > 0, z/i,fc > 

with x{k) a Gaussian random vector with expectation E(a;(A;)) and covariance matrix 
cov{x{k),x{k)). 

Neuronal pulse trains approximating Young measure solution. At each time t, the 
measure r/^* can be approximated by the fraction of the neuron ensemble that are firing. In 
detail, assuming that the means of the optimal control signals are Rt^i{^), i = 1,2, and 
there are one ensemble of excitatory neurons and another ensemble of inhibitory neurons. 
A fraction of the neurons fire so that the mean firing rates satisfy: 

x:{t) = ^{E[Rrit)]-E[Rr\t)] 

where Rf'^^{t) and Rl^^{t) are the firing rates of the excitatory and inhibitory neurons 
respectively, and 7 is a scalar factor. In occurrence of sub-Poisson noise, the noisy control 
signals u*{t) = A*(t) + Q(t) are approximated by 

um = ^[Rnt) - Rr\t)]. 

Both ensembles of neurons are imposed with baseline activities, which bound the minimum 
firing rates away from zeros, given the spontaneous activities of neurons when no explicit 
signal is transferred. A numerical approach involves discretise time t into small bins of 
identical size At. The firing rates can be easily estimated by averaging the population 
activities in a time bin. We have used 400 neurons to control the system, with two ensembles 
of neurons with equivalent numbers that approximate the first and second components of 
control signal, respectively. Each neuron ensemble have 200 neurons with 100 excitatory and 
100 inhibitory neurons. 

Appendices 

Appendix A: Derivation of formula (H]) 

Let: W be the time-varying p.d.f. p(x, t) that is second-order continuous -differentiable with 
respect to x and t that is embedded in the Sobolev function space Pl^^'^; Wi be the function 
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space of j9(x, to)^ regarding as a function with respect to x with a fixed tg; W^2 be the function 
space of p(xo, t), regarding as a function with respect to t with a given xq. The spaces W^i 2 
can be regarded being embedded in W . In addition, let: W be the function space where the 
image C\W] is embedded; L be the space of linear operator £, denoted above; ^{Z, E) be 
the space composed of bounded linear operator from linear space Z to E\ and Z* be the dual 
space of the linear space Z: Z* = ^(Z, M). Furthermore, let y be the tangent space of 
Young measure space y: Y = {rj — r]' : r],r]' E y}. For simplicity, we do not specify the 
spaces and just provide the formalistic algebras, and then the following is similar to Chapter 
4.3 in lfT2l with appropriate modifications. 
Define 

^{p) = / \\(f){x,t) - z{t)fp{x,t)dxdt 

n(p, v) = (-^ - (>C ■ 77) o p, p{x, 0) - po{x)j 

m ^/^,(MWMM.-.w. (.7) 

Thus, dll) can be rewritten as: 

subject to n(p, rj) = 0, J{p) = 0. 
The Gateaux differentials of these maps with respect to p{x, t), denoted by Vp-, are: 

{Vp^)o{p-p)= / J<j){x,t)-z{t)\\^[p{x,t)-p{x,t)]dxdt 

[dip — p) 
(VpH) o{p-p) = [^ — — (>C-?7) o(p-p),p(x,0) -p(x,0) 



(VpJ)o(p-p) = / (t){x,t)\p{x,t)-p{x,t)]dx 



for two time-varying p.d.f. p,p G W. Here, Vp$ G VT*, VpII e ^(VT, VT x VTi), 
VpJ E J^{W, W2). And, the differentials of these maps with respect to the Young measure t] 
are: 

(V,$) .{fi-r])=0 

(V,n) ■ (r) - r^) = ((£op).(r}-r/),0) 

(V,J) •(//-//) =0 

for two Young measures r/,?7 G y. Here, V,,$ G 3^*, V^H G ^{y,W x H^i), and 

V,Jg^(>^,W^2). 

Then, we are in the position to derive the result of dS) by the following theorem, as a 
consequence from Theorem 4.1.17 in [|T2l . 



Theorem 1 $ : VT ^ VT*, n : VT x 3^ ^ M an J J : W ^ W2 as defined in (UZp. A^^Mme 
that: (1). the trajectory ofx{t) in (I?)) is bounded almost surely; (2). a{x, t), b{x, t) and (j){x, t) 
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are C^ with respect to (x, i). Let {ri*,p*) be the optimal solution o/®- Then, there are some 

Ai G ^{W X Wi, W*), X2 = [A21, A22]^ with A21 G ^{W, W2), and A22 G ^{Wi, , 1^2), 

Ai o VpUip*,r]*) = Vp^p*), A2 o VpU{p*,rj*) = VpJ{p*), (19) 

and the abstract maximum principle 

r/*|minimaof /i(t,0 w.r.t{} = 1, Vt G [0,T + i?]. (20) 

holds with "abstract Hamiltonian": 



h{t. 



0= - /p*(x,t){X^A,(x,t,0|^ 

+ - $^[S(x,t,Oi?(x,t,erk^^}^^- (21) 

Proof. Under the conditions in this theorem, we can conclude that the Fokker-Planck 
equation has a unique solution p(r]) that is continuously dependent of 77 from theory of 
stochastic differential equation [|T71 : n(-, 77) : W — ;■ W* is Frechet differentiable at p = 7i(r]) 
because x{t) is assumed to be almost surely bounded; n(p, ■) : y — )■ W* and J (in fact 
V;y J = 0) is Gateaux equi-differentiable around p E W because ofp E W C W"^'^ with (x, t) 
bounded t26\; the partial differential Vjyll is weak-continuous with respect to 77 because it is 
linearly dependent of 77. 

In addition, from the existence and uniqueness of the Fokker-Planck equation, 
Vpll{p,ri) : W — )■ Im{\/pll{p,ri)) C W x Wi has a bounded inverse. This implies that 
the foUowingaJ/om? equation 

p o VpII = Vp<l> + c(t) o Vp J, 

has a solution for p = p{r]), denoted by yU. Let /i = [fii, P2], which should be solution of the 
following equation 

^ + (£* ■ r/*) 0^1 = -\\<p{x,t) - z{t)r - c{t)<p{x,t) 

/i2(x) =/ii(x,0) (22) 

/ii(x,r + /?) = t G [o,r + i?], xgh, 

with the dual operator C* of C (the operator in the back-forward Kolmogorov equation), still 
dependent of (x, t) and the value of A(t) (namely ^ in Young measure): 

Wepick Ai^2(^) with//i = Ai + c(t)A2i andyU2 = Ai + c(t)A22- So, Ai_2 should satisfy equation 
(fT9l) . In fact, Ai 2 can be regarded as functions ( or generalized functions) with respect to (x, t) . 
Thus, the conditions of Lemma 1.3.16 in [[T2ll can be verified, which implies that the 
gradients of the maps $ and J with respect to r], by regarding p = p{r]) from Il{p, r]) = 0, as 
follows: 

5$ = v„$ - Ai o v„n, dj = v„ J - A2 o v„n. 
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From the abstract Hamilton minimum principle (Theorem 4. 1 . 17 in lfT2]| ). applied to each 
solution of 111), denoted by r/*, there exists a nonzero function c(t) such that 

H{f,) = 9$(r/*) ■ fj + (c(t), dJ{r]*) ■fi),^fjey (23) 

is an 'abstract Hamiltonian', with respect to 77. With the definitions of Ai 2, (1231) becomes 

H{fj) = Wr,^ ■ fi + cW^J -v-iiJ., V^n ■ r/) = -(/i, v^n ■ fj), 

owing to Vr,$ = V^ J = 0. 

By specifying fx with Ai^25 we have 

H{fj) = - (/i, V,n ■ fj) = -(/ii, [C o p*] . fj) = -{p*, [C* o /ii] . r/) 



i /J/-{s:-(-«)|r 



«,J=1 -^ 

where p* stands for the time-varying density corresponding to the optimal Young measure 
solution T]*. From this, letting ^ = A, we have the "abstract Hamiltonian" in the form of (1211 
as the Hamiltonian integrand of H{-). 

The Hamiltonian abstract minimum (maximum) principle indicates the optimal Young 
measure rj^ is only concentrated at the minimum points of h{t, ^) with respect to ^ for each t, 
namely. That is, (|20|) holds. This completes the proof. ■ 

From this theorem, since the variances depend on the magnitude of the signal as 
described in (|3]), removing the terms without ^, it is equivalent to look at the minima of 
h{t, ^) in the form of 

m 

h{t,0 = 5^/i.(t,e*), h,{t,C,) = ^.(t)|6|'" - Mm (24) 

i=l 

instead of h{t, ^), where 

/»W = 5Z / P*ix,t)bji{x,t)—^dx, 

9iif) = ^ Y X^ / P*ix,t)bji(x,t)bkiix,t) ^ dx. 
This gives formula ([8]). 

Appendix B: Derivation of precise control performance (O 

The control performance inequality (flBT l can be derived from the following theorem. 

Theorem 2 Let x be the solution of equation ([77]) and < a < 0.5. Assume that there 
are a positive measurable function K{t) and a positive constant Ci such that ||A(x, t, u) — 
A{y,t,u)\\ < K{t)\\x-y\\, \\B{x,t,u)f < ft:(t) ^^l^ I^^P" anJ|0(x, t) - 0(i/, t)| < 
Ci||x — y\\ hold for all x,y G M" and t > 0. Then, for any non-random initial value, 
namely, x(f)) = E(x(0)), with the non-optimal Young measure ( 172]) . we have 



2-Q 

Y 
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(i) j™\\E{x{t))-z{t)\\^0; 
(ii) min^ ^Jf™v&T{x)dt = 0(^^ 

as My -^ oo. 

Proof. Comparing the differential equation of x, i.e. (01), and that of x, (fTTI) . we have 
d{x — x) = [A{x,t,u) — A{x,t,u)\dt + B{x,t,\{t))dWt. And, replacing A (t) with the Young 
measure i7t(-), in the form of (fT2l) . from the conditions in this theorem, we have 

E||x(r)-x(r)f = e| j [A{x{t),t,u) - A{x{t),t,u)]dtV 
+ e| /" ||5(x,t,A)f -r^irft} 

<e| /" «:(t)||x(t)-x(t)||dt|' + V f\{t)\\u\^'' ■ f,u,tdt 

< K\s)ds E\\x{t)-x{t)\\^dt + J2 J^<t)\Uk{t)\dt, 

•-' U •-'0 jr. 1 •/ ^ 

for any r > 0. By using Gronwall's inequality, we have 

rT+R pT+R 

/ E||x(r)-x(r)frfrrft< / E\\x{t) - x{T)fdrdt < 
Jt Jo 

pT+R pT+R PS 1 1 "* 

/ exp / / K^{T)dTds —YZ2^y^K.{t)\uk{t)\dt. (25) 

JQ '-Jt Jo -i My ^^-^ 

Noting that for < a < 0.5, HmMy^oo 1/My~^° = implies that /^ "^^E||a;(r) - 
x{T)\\'^dTdt = 0(1/My~^") as Mygoes to infinity. This proves the second item in this 
theorem. 

In addition. 



/I +U / Pi +H 

\\E{xit))-zit)\\<VRjEJ^ {\\xit) - xitWdt} 

also approaches zero as My goes to infinity. This proves the first item of the theorem. This 
completes the proof. ■ 

Hence, as My goes to infinity, the non optimal solution (fT2)) can asymptotically satisfy 
the constraint and the error variance goes to zero as My goes to infinity. Therefore, the 
performance error of the REAL optimal solution of the optimisation problem @ approaches 
zero as My — )■ oo in the case of a < 0.5. Furthermore, we can conclude from (^5) that the 
execution error, measured by the standard deviation, can be approximated as (fT3T ). 
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Figure 1. Illustration of possible minimum points of the function gi(i)|^ip" — fi{t)^i in 
hi{t,^i) with respect to the variable ^,j with g.i{t) = 1, /^(t) = 2 and My = 10 for 
a = 0.8 > 0.5 (blue curves) and a ~ 0.25 < 0.5 (red curves): (a), the plots of 
3i(*)l6P" ~ /i(06' with respect to ^,; for a = 0.8 (blue) and a = 0.25 (red) and their 
mimimum points; (b) the inner plot of gi{t)\(,i\'^°' — fi{t)£,i for ^i G [0, 0.2] to show that the 
^j = does be a minimum point for a = 0.25 (red); (c). the plots of the derivatives of 
9ii't)\^i?°' - fi{i)it with respect to ^^ for a = 0.8 (blue) and a = 0.25 (red). 




H=(x^(t),X2(t)) 



Q 



Figure 2. Illustration of the arm control. Arm is composed of three points (P, Q, and H), 
where P is fixed and others not, and two arms (upper arm PQ and the forearm QH). Button 
H is to reach some given target (red cross) by moving front- and back-arms. 



Table 1. Summary of the possible minimum points of (|8). 

g,{t) > g^{t) < 



a > 0.5 one point in [—My, My] 
a<0.5 {0, My} or {0, -My} 



{My} or {-My} 
{My} or {-My} 
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Figure 3. The means of the optimal control signals Ai (i) (blue solid) and A2 (green solid) in 
the straight-trajectory arm movement example with a = 0.25 and My ~ 20000. The blue 
and red dash vertical lines stand for the start and end time points of the duration of reaching 
the target respectively. 
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Figure 4. Optimal control of straight-trajectory arm movement model with parameters Usted 
in Table |2] Target is set by Oi{T) = — 1 and 02{T) = ^ but without noise: The dynamics of 
the angles (a), the angle velocities (b) and accelerations (c) (the blue solid curves for those of 
01 and the green solid curves for 6*2). The blue and red dash vertical lines stand for the start 
and end time points of the duration of reaching the target. 
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Table 2. Parameters. 



Parameters 



Values 



masses (of the inertia w.r.t the mass center) 

lengths (of the inertia w.r.t the mass center) 

moments (of the inertia w.r.t the mass center) 

lengths of arms 

reach time 

duration 

target 

scale parameter 

noise scale 

bound of the control signal 

time step 



mi = 2.28 kg, m2 = 1.31 kg 

k = 0.305 m, h = 0.254 m 

h = 0.022 kg ■ m^\ h = 0.0077 kg ■ m^ 

ri = 0.133 m, r2 = 0.109 m 

T = 650 ms (except in Figs. |7] and [8]) 

R = 10 ms (except in Figs. |7] and [8]) 

ei(T) = -l,^2(T) = 7r/2 

ro = 1 

Ko = 1 

My = 20000, except in Figs. [Hand [6] 

and the inset plot of Fig. |9] 

At = 0.01 ms 
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Figure 5. Optimal control of straight-trajectory arm movement model with noise and the same 
model parameters as in Fig. |4] and a = 0.25, My = 20000: The dynamics of the angles (a), 
the angle velocities (b) and accelerations (c) (the blue solid curves for those of 9i and the red 
solid for 02)- The blue and red dash vertical lines stand for the start and end time points of the 
duration of reaching the target. 
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Figure 6. Optimal control signal of Young measure of straight-trajectory arm movement 
model with noise, illustrations for Ai (a) and A2 (b) in discrete-time way with My = 5, 
where width of each bar stands for measure of Aly at each time bin. 
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Figure 7. Means of the optimal control signals Ai (i) (blue solid) and A2 (i) (green solid) in the 
straight-trajectory arm movement example with T = 100 (sec) and i? = 100 (msec). The blue 
and red dash vertical lines stand for the start and end time points of the duration of reaching 
the target. 
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Figure 8. Optimal control of straight-trajectory arm movement model with noise with 
T = 100 and R = 100 (msec): The dynamics of the angles (a), the angle velocities (b) 
and accelerations (c) (the blue solid curves for those of di and the red solid curves for 6*2). 
The blue and red dash vertical lines stand for the start and end time points of the duration of 
reaching the target. 
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Figure 9. Performance of optimal control of straight-trajectory arm movement model: 
Relationship between the executive error, measured by mean standard variance, and dispersion 
index a with My ~ 20000 and Log-log plot (the inner plot) of the the relationship between 
executive error and bound of the Young measure My with a = 0.25 where the dash line is 
reference line with slope —1/2 + a — —0.25, as shown in ( fT3] l. 
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Figure 10. Spiking control of straight-trajectory arm movement model: (a). Approximation 
of first component {u\{t)) of the optimal Young measure control signal by spike trains; (b). 
Approximation of second component (itj (i)) of the optimal Young measure control signal by 
spike trains; (c) and (d). Approximation of optimal control signal Q 1,2 by spike trains of a set 
of neurons. 



